Pruning-aware Sparse Regularization for Network Pruning

Structural neural network pruning aims to remove the redundant channels in deep convolutional networks (CNNs) by filters of less importance final output accuracy. To reduce degradation performance after pruning, many methods utilize loss with sparse regularization produce structured sparsity. In this paper, we analyze these sparsity-training-based and find that unpruned is unnecessary. Moreover, it restricts network’s capacity, which leads under-fitting. solve problem, propose a novel method, named MaskSparsity, pruning-aware regularization. MaskSparsity imposes fine-grained on specific selected mask, rather than all model. Before MaskSparity, can use get such as running global achieves 63.03% float point operations (FLOPs) reduction ResNet-110 removing 60.34% parameters, no top-1 accuracy CIFAR-10. On ILSVRC-2012, reduces more 51.07% FLOPs ResNet-50, only 0.76% The code paper released at https://github.com/CASIA-IVA-Lab/MaskSparsity . We have also integrated into self-developed PyTorch toolkit, EasyPruner, https://gitee.com/casia_iva_engineer/easypruner